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ABSTRACT 

This study examined the effects of taking notes in 
the portion of the Test of English as a Foreign language (TOEFL) 
listening comprehension section that contains short monologues, or 
*'mini-talks . " These effects were assessed in experimental testing 
sessions with students in intensive English language programs and 
with undergraduate and graduate international students (n=563) . A 
multiple-choice questionnaire surveyed the students' reactions to the 
opportunity to take notes and their previous note-taking experience. 
Allowing students to take notes had little effect on their 
performance, and urging students to take notes significantly impaired 
their performance. These effects were observed even for students who 
reported being in the habit of taking many notes or reported having 
classroom instruction in note taking. Apparently, then, little 
benefit is gained by taking notes in the context of the present TOEFL 
mini-talks, perhaps because they are designed to assess listening 
comprehension wi^h minimal demand placed on memory. Responses to the 
questionnaire aid in understanding the results and provide useful 
general information about the students' note-taking experiences and 
habits. Seven tables precent study data, and an appendix gives a 
sample mini-talk script. (Contains 15 references.) (Author/SLD) 
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Abstract 



This study examined the effects of taking notes in the portion of the 
TOEFL listening comprehension section that contains short monologues, or 
"mini- talks." These effects were assessed in experimental testing sessions 
with students in intensive English language programs and with undergraduate 
and graduate international students. A multiple-choice questionnaire 
surveyed the students' reactions to the opportunity to take notes and their 
previous note-taking experiences. 

Allowing students to take notes had little effect on their performance, 
and urging students to take notes significantly impaired their performance. 
These effects were observed even for students who reported being in the 
habit of taking many notes or reported having had classroom instruction in 
note taking. Apparently, then, little benefit is gained by taking notes in 
the context of the present TOEFL mini -talks, perhaps because they are 
designed to assess listening comprehension with minimal demand placed on 
memory. Responses to the questionnaire aid in understanding the results and 
provide useful general information about the students' note-taking 
experiences and habits. 
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Introduction 



Objectives of the Study 

One of the purposes of the Test of English as a Foreign Language 
(TOEFL®) is to assess students' ability to comprehend spoken English as it 
would typically occur in an academic setting. Toward this end, the listening 
comprehension section of the TOEFL test contains several parts, one of which 
consists of short monologues, called "mini- talks , " followed by questions about 
them. 

Students are not allowed to take notes, because the mini- talks and 
questions are designed in such a way that the student who comprehends what is 
said presumably should be able to answer the questions correctly without 
taking notes. Nevertheless, some observers have suggested that students be 
allowed to take notes, arguing that this would make the test more comparable 
to the typical academic listening situation. In light of this suggestion, the 
present study assessed the effects of note taking on performance in mini- talks 
taken from the TOEFL test. 

To determine the merits of note taking, it is of value not only to 
examine its effects on average performance, but also to look at the 
correlation between test performance under note- taking and non-note - taking 
conditions. A low correlation (relative to test reliability) would show that 
allowing note taking affects the relative standing of students on the test 
and, to that extent, alters the character of the test. 

In addition, it is useful to ascertain students' opinions about being 
able to take notes when listening to mini -talks. Among issues of concern are 
whether the students feel that being allowed to take notes is beneficial and 
whether their perception is affected by their previous note- taking experiences 
and habits. It is also of interest to determine whether students who report 
being most accustomed to taking notes are particularly likely to show positive 
effects of note taking on performance. 

To address these icsues, students listened to some mini-talks with the 
opportunity to take notes and listened to other mini-talks without being able 
to take notes (or the reverse order, depending on the group assignment.) The 
students then completed a multiple - response questionnaire asking for their 
opinions about the present note-taking experience and inquiring into their 
prior note- taking habits. 

One other issue of interest concerned the way iii which the note- taking 
situation was structured. The primary question of the study was how 
performance would be affected if students were simply allowed to take notes, 
given that the typical academic listening situation calls for letting student^: 
take notes or not, as they choose. An additional question was how performance 
would be affected if students were explicitly asked to take notes, as this 
question bears on the extent to which the actual act of taking notes would 
influence performance. Thus, students in the principal condition were simply 
permitted to take notes, whereas students in another condition were expressly 
asked to take notes. These two conditions are termed the "Note Taking 
Allowed" and "Note Taking Urged" conditions, respectively. In each condition, 
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a comparison was made between a test situation involving note taking and one 
involving no note taking. 



Relevant Literature 

In the literature there has been an emphasis on two general functions 
of note taking: encoding and external storage (cf. DiVesta & Gray, 197?; 
Fisher & Harris, 1973). Encoding refers to the role of note taking in 
ensuring that the lecture information is properly understood and coded into 
memor> This can include increasing the student's attention to the lecture 
(Frase, 1970), increasing awareness of the organization of the talk (Dunkel & 
Pialorsi, 1982; Einstein, Morris, & Smith, 1985), inducing the student to 
compare the information heard with prior knowledge (DiVesta 6f Gray, 1972), and 
other such influences. External storage, on the other hand, typically refers 
to the role played by the notes in reviewing for a test given after a delay. 

If note taking were allowed in connection with the TOEFL mini- talks, its 
primary function would involve encoding, or processes that occur at the time 
the notes are taken. The external storage, or review, function of notes would 
be minor, primarily because the test is given immediately, thus ruling out any 
substantial benefit associated with the use of notes to forestall long-term 
forgetting.^ Thus, the studies in the literature that appear most relevant 
to the present research are those dealing with the encoding function of note 
taking, where an immediate test is used. And of those .studies , the ones that 
relate most directly to the principal issue here- -namely , whether the 
opportunity to take notes affects performance -- are studies that have included 
a comparison between note- taking and non-note- taking situations. 

Some such studies have found that permitting note taking has a positive 
effect under certain circumstances (e.g., Barnett, DiVesta & Rogozinski, 1981; 
DiVesta & Gray, 1972: 1973; Einstein et al . , 1985). Other studies have failed 
to find an effect (e.g.. Carter & Van Matre, 1975; Dunkel, 1986). And still 
others have observed an interfering effect (e.g., Aiken, Thomas, & Shennum, 
1975; Peters, 1972). Although reasons for the differences in results are not 
entirely obvious, a general conclusion is that the opportunity to take notes 
does not necessarily produce beneficial effects. Rather, the effects of being 
able to take notes may depend heavily on the conditions under which note- 
taking effects are assessed, as will be considered further in the Discussion 
section. 

With the exception of the study by Dunkel (1986), the research just cited 
has involved only native English speakers (although other issues related to 



^Granted, students would need to be allowed to keep their notes before 
them while answering the questions (which was the case in this study), as i 
would not be practical to withdraw ;he notes after each mini -talk in this 
type of test. Nevertheless, because uhe test is given Immediately, the 
primary benefit of note taking in this case is believed to be linked to 
processes occurring during the time the notes are taken, or encoding 
processes . 
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note taking have been studied with nonnative English speakers). It is 
possible that the effects of note taking may differ for nonnative English 
speakers; as Dunkel and Davey (1989) note, these students process aural 
information in English more slowly than do native English speakers, and they 
may also differ in note- taking experiences and habits. Still, results of 
these studies provide a useful framework for interpreting the results of the 
present research with nonnative English speakers. 
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Method 



Subjects 

A total of 563 international students were tested, 288 students in four 
intensive English programs and 275 students, both graduate and undergraduate, 
enrolled in academic coursework in three universities. The students were 
invited to participate and were offered $20 to do so. A total of 69% of the 
students were college graduates. All were nonnative English speakers. Native 
languages of the students were Chinese, including several major dialects (with 
Chinese speakers comprising 27% of the total sample) , Japanese (18%) , Spanish 
(8%), Korean (6%), Arabic (5%), French (5%), and 39 other language groups with 
3% or fewer students in each. Native countries represented were the People's 
Republic of China (22% of the sample), Japan (18%), the Republic of Korea 
(6%), Taiwan (5%), and 65 other countries with 3% or fewer students from each. 



Materials 

Mini" talks . Tho six mini -talks used in the study were drawn from 
previously disclosed TOEFL tests. The mini- talks ranged in length from 180 to 
250 words and were presented at an average pace of 145 words per minute. Each 
mini-talk was presented on tape and was followed by five questions, also 
presented on tape, with response options appearing in the test booklet. The 
mini- talks were separated into two groups. The three mini- talks presented 
first, with their accompanying questions, were labeled Test A , and the three 
mini-talks presented second were labeled Test B . Prior to beginning Test A, 
the students were given two "warm-up" questions based on a condensed mini- 
talk. 

Although the mini-talks required no prior knowledge in any subject-matter 
area, they could nevertheless be classified according to whether the topic 
related to (a) the humanities or social sciences, (b) the biological or 
physical sciences, or (c) a subject of general interest. One mini-talk of 
each type was used in Test A, and one of each type was used in Test B. The 
appendix shows one of the mini -talks used in the study. (The full set of six 
mini-talks cannot be presented, as many of them must remain secure for use in 
another context.) In the script, the abbreviations MA and MB refer to "Man A" 
.nd "Man B," the two people who recorded the mini -talk and questions. Of the 
six mini- talks, three were recorded by a man- -two in Test A and one in 
Test B--and three were recorded by a woman. 

Qu estionnaire . In the questionnaire, the students were asked for their 
reactions to the present note-taking situation and about their prior note- 
taking experiences and habits. The questionnaire consisted of two parts. In 
the first part, the students were given several statements and told to 
indicate the extent to which they agreed with each. Four response options 
were provided: "I strongly disagree," "I disagree," "I agree," and "I 
strongly agree." In the second part of the questionnaire the students were 
asked several questions, and for each question the students checked the 
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appropriate response option. The statements and questions appearing in the 
questionnaire can be seen in Tables 5, 6, and 7 of the Results section. 



Experimental Design 

Table 1 depicts the study's design. Each student received Test A 
followed by Test B. There were four groups: (a) Note Taking Allowed 
condition, Order 1; (b) Note Taking Allowed condition, Order 2; (c) Note 
Taking Urged condition, Order 1; and (d) Note Taking Urged condition. Order 2. 
In the Note Taking Allowed condition, note taking was simply permitted during 
one of the two tests, whereas in the Note Taking Urged condition, students 
were requested to take notes during one of the two tests. For Order 1 in each 
case, the note- taking situation was associated with Test A, and the non-note- 
taking situation, with Test B; whereas for Order 2, the reverse was true-- 
i.e., the non-note - taking situation was associated with Test A, and the note- 
taking situation, with Test B. 

Inclusion of both intensive English language students and academic 
students in each experimental group was intended to ensure representation of a 
wide range of proficiency in each group, comparable to the range found among 
TOEFL examinees. 



Procedure 

The students were tested in groups by a test administrator (author GH) 
and a monitor (author RC) . The general test-taking instructions were printed 
on the students' test booklets and read aloud by the test administrator; 
additional test- taking instructions were presented via tape recorder. The 
specific instructions regarding note taking were presented via tape recorder 
and the essential points were repeated by the test administrator. (See 
Instructions , below. ) 

As noted above, the students received Test A (i.e., first three mini- 
talks) followed by Test B (the next three mini-talks), with note-taking 
associated with oiie test, and non-note- taking associated with the other, the 
order of assignment depending on the student's experimental group. For each 
question, the students responded by locating the correct answer in the test 
booklet and marking the letter corresponding to the correct answer on a 
separate answer sheet. Twelve seconds were allowed for responding to each 
question. The students were allowed to keep their notes before them while 
answering the questions. Prior to beginning Test A, the students were given 
the two warm-up questions based on a condensed mini -talk in order to provide 
practice in finding the correct answer and marking the answer sheet. After 
finishing Tests A and B, the students filled out the questionnaire. The whole 
session lasted approximately 50 minutes. 

Instructions . Shown below are the tape-recorded instructions for each 
condition of the study; portions marked by brackets were repeated by the test 
administrator after presenting the tape-recorded instructions. 
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Table 1 
Experimental Design 



Note Taking Allowed Condition 

Order 1 (Note taking for Test A; no note taking for Test B)* 

N - 148 (79 students from English language program 1; 
69 academic students from University 1) 

Order 2 (No note taking for Test A; note taking for Test B) 

N 138 (66 students from English language program 2; 
72 academic students from University 2) 



Note Taking Urged Condition 

Order 1 (Note taking for Test A; no note taking for Test B) 

N - 130 (68 students from English language program 3; 
62 academic students from University 2)^ 

Order 2 (No note taking for Test A; note taking for Test B) 

N - 147 (75 students from English language program 4; 
72 academic students from University 3) 



^Students in all groups received Test A (3 mini- talks, 15 total items) 
followed by Test B (3 different mini- talks. 15 total items). 

^University 2 contributed academic students to each of two different 
experimental groups 
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In the non-note- taking situation students were given the following 
instructions : 

[You may not take notes when you listen to the talks in this part of the 
test. Also, you may not write in your test book. Remember, you may not 
take notes when you listen to these talks.] 

In the Note Taking Allowed condition, students were given the following 
instructions before the test assigned to the note-taking situation: 

[If you wish, you may take notes when you listen to the talks in this part 
of the test.] You may take notes either in English or in another language. 
[In your test book are special pages on which to write your notes.] These 
are the pages with the word "Notes" at the top. [Your notes will not be 
graded. We want to see how note taking might help people to take the test. 
So remember that, if you wish, you may take notes when you listen to these 
talks. You may not write on any other page of your test book.] 



In the Note Taking Urged condition, students were given the following 
instructions before the test assigned to the note -taking situation: 

[Please take notes as you listen to the talks in this part of the test.] 
You may take notes either in English or in another language. Also, you may 
write many notes or just a few notes. But please be sure to write some 
notes. [In your test book are special pages on which to write your notes.] 
These are the pa ges with the word "Notes" at the top. [Your notes will not 
be graded. We want to see how note taking might help people to take the 
test. So please be sure to take notes when you listen to these talks. You 
may not write on any other page of your test book.] 
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Results 



Performance Effects 

Note Taking Allowed . The situation of primary interest in the study was 
the Note Taking Allowed condition. Table 2 presents the mean test scores in 
this condition for each order. It is apparent that Test B, presented second 
to all students, yielded higher scores than did Test A, presented first. This 
difference, in itself, was not of concern here (and could have been due either 
to differences in test difficulty or to a practice effect). The important 
question was whether this difference was significantly greater (or less) for 
Order 2 than for Order 1, according to the analysis described below. If note 
taking were to have a positive effect, it would be manifested as a greater 
superiority of Test B over Test A for Order 2 than for Order 1. 



Table 2 

Note Taking Allowed Condition: 
Means and Standard Deviations (in Parentheses) of Test Scores 



Test A Test B 



Order 1 (N - 148) 

(Note taking allowed for Test A; 7.95 8.65 

no note taking for Test B) (2.96) (3.3 7) 

Order 2 (N - 138) 

(No note taking for Test A; 10.56 11.50 

note taking allowed for Test B) (2.86) (2.82) 



An analysis of variance was performed, in which the between- subj ect 
factors were Order and Educational Status (i.e., intensive English vs. 
academic students) , and the dependent variable was the difference between Test 
A and Test B . No effect even approached significance. Ac discussed above, 
the influence of note taking is reflected in the Order effect. The fact that 
this effect was nonsignificant, F < 1, shows that being allowed to take notes 
had little Influence on the students' performance. Also, the fact that 
Educational Status did not interact significantly with Order shows that note 
taking was no more likely to affect performance for the academic students than 
for the intensive English students. 

Note Taking Urged . Table 3 presents the mean scores for students in the 
Note Taking Urged condition. In this case, the difference in favor of Test B 
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over Test A was considerably less for Order 2 than Order 1, showing that when 
the students were urged to take notes, their performance was actually 
impaired. This conclusion was reinforced in an analysis of variance 
comparable to that performed for the Note Taking Allowed condition. In this 
case the Order effect was highly significant, F (1,273) - 20.83. £<.001, 
indicating a negative effect of being urged to take notes. Other effects were 
nonsignificant, including the interaction between Order and Educational 
Status, showing that the negative effect of urging the students to take notes 
was equally great for intensive English students and academic students. 

Table 3 

Note Taking Urged Condition: 
Means and Standard Deviations (in Parentheses) of Test Scores 



Test A Test B 



Order 1 (N « 130) 

(Note taking urged for Test A; 9.38 10.92 

no note taking for Test B) (3.17) (3.01) 

Order 2 (N - 147) 

(No note taking for Test A; 10.17 10.35 

note taking urged for Test B) (3.46) (3.30) 



Test Reliabilities and Correlations . Analyses were performed to 
determine, for each subgroup, the coefficient-alpha reliabilities of Test A 
and Test B, and the correlation between Test A and Test B. The results of 
these analyses are presented in Table 4. 

Table 4 

Test Reliabilities and Correlations 



Reliability Correlation 
Test A Test B Between Tests 



Note 


Taking Allowed, Order 1 


.68 


.72 


.63 


Note 


Taking Allowed, Order 2 


.71 


.74 


.75 


Note 


Taking Urged, Order 1 


.74 


.75 


.68 


Note 


Taking Urged, Order 2 


.81 


.78 


.74 
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The reliabilities of the two tests were relatively high, even though each 
test contained only 15 items. So the tests apparently provided a reasonably 
sound basis for measurement of note-taking effects. Also, there was no clear 
difference between tests with respect to reliability. Of particular note, the 
correlations between tests were nearly as large as the reliability 
coefficients, so that the correlations corrected for attenuation were all 
above .90. This suggests that the relative standing of students was not 
markedly different for the note-taking and non-note - taking situations. 

Questionnaire Data 

Questions about the present testing situation . Data on the students' 
responses to the questionnaire items are presented in Tables 5, 6, and 7. For 
the statements appearing in Tables 5 and 6, the students had been given four 
response options strongly disagree," "disagree," "agree," and "strongly 
agree." However, the data were combined across the two "agree" categories and 
across the the two "disagree" categories, as inspection of the data broken 
down by all four categories did not appear to alter the general conclusions. 
The proportion of students agreeing with each statement are presented. The 
data are combined for Orders 1 and 2, as this factor should have little 
bearing on the responses. 

Of initial interest were the data in the top half of Table 5, which 
indicate the students' general react:ions to the present note- taking 
experience. A small majority of students agreed with the statement, "Taking 
notes helped me to ansv;er the questions better." There was also a moderate 
tendency for the students to report feeling more at ease when they could take 
notes. To the extent that the students felt it advantageous to take notes, 
the reported advantage appeared to lie in helping the students remember the 
information in the talks, rather than helping them to listen carefully or to 
understand the talks. It is not clear v;hy fully three - fourths of the students 
believed note taking helped them remember the information in the talks. It 
appears that there was a discrepancy between students' perceptions and 
reality, in that the students were unaware that note taking, on average, did 
not help them answer the questions better in this situation. 

The items at the bottom of Table 5 further help to explain the students' 
feelings about the value of note taking in the present situation. There was 
widespread sentiment that there was not enough time for taking notes. There 
was substantial agreement that taking notes would have helped more if the 
questions had asked about details, such as names and dates. And there was 
substantial disagreement with the statement that the talks were too easy for 
note caking to help. For the other statements there was no strong tendency 
toward agreement or disagreement. 

Questions about note- taking experiences and habits . The students were 
asked not only about note taking in the present situation, but also about 
their general note- taking habits and experiences. This was done partly to 
permit analyses of performance effects separately for groups differing i.u 
these respects, and also to obtain general information regarding students' 
perceptions of the value of note taking. 
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Table 5 

Questionnaire Items about Present Note -Taking Experience: 
Proportion of Students Agreeing with Each Statement 



Note Taking Note Taking 

Allowed Urged 



GENERAL STATEMENTS ABOUT THE VALUE OF NOTE TAKING 



Taking notes helped me to 



answer the questions better. 


.56 


.57 


Taking notes made it more 






difficult to understand the talks. 


.39 


.46 


I felt more at ease when I could 






take notes than when I could not. 


.64 


.61 


Taking notes made the test 






more difficult for me. 


.33 


.37 


Taking notes helped me to 






listen carefully to the talks. 


.45 


.37 


Taking notes helped me to 






understand the talks. 


.48 


.39 


Taking notes helped me to remember 






the information in the talks. 


.77 


.72 



STATEMENTS ABOUT SPECIFIC ASPECTS OF NOTE TAKING 



I wanted more time to review my notes 






before answering the questions. 


.51 


.53 


I actually used my notes 






when I answered the questions. 


.49 


.50 


It was difficult to switch from taking 






notes to marking my answer sheet. 


.47 


.49 


I had enough time to take 






as many notes as I wanted. 


.18 


.15 


The questions were about things 






I had written in my notes. 


.51 


.56 


The talks were too short for 






note taking to help me very much. 


.46 


.46 


The talks were too easy for 






note taking to help me very much. 


.21 


.21 


Taking notes would have helped me more 






if the questions had asked about 






details such as names and dates. 


.83 


.82 
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Table 6 

Questionnaire Items about American Classroom Note -Taking Experiences: 
Proportion of Students Agreeing with Each Statement 
(Academic Students Only) 



Note Taking Note Taking 
Allowed Urged 



Taking notes is more helpful in American 

class lectures than in the test I took today. ,92 .93 

Taking notes in American class, lectures 

helps me get better exam marks. ,90 ,93 

In American class lectures I usually have 

enough time to take as many notes as I want. .58 .53 

Taking notes helps me listen carefully 

to American class lectures, .67 .56 

Taking notes helps me to understand 

American class lectures. .80 .73 

Taking notes helps me remember the 

information in American class lectures. .96 .95 

Taking notes in an American class lecture 
helps me to organize the information after 

the lecture is over. .95 .96 

Taking notes in American class lectures gives 

me useful information to study for exams. .97 .96 
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Table 7 



Questions about Note-Taking Instruction and Habits: 
Proportion of Students Giving Each Response 



Note taking 
Allowed 



Note Taking 
Urged 



How many notes do you None 

usually take in an Few 

American class lecture? Some 

(academic students only) A lot 



In your native 
country, how many 
notes did you 
usually take in 
class lectures? 



None 
Few 
Some 
A lot 



.00 
.10 
.46 
.40 



.04 
.14 
.33 
.50 



.01 
.09 
.53 
.33 



.02 
.14 
.40 
.44 



How many notes did 
you take on the test 
today? 



None 
Few 
Some 
A lot 



.26 
.57 
.15 
.01 



,05 
.68 
.23 
.03 



In what language do you 
usually take notes in an 
American class lecture? 
(academic students only) 

In what language did 
you take notes on the 
test today? 

Have you had any 
classroom instruction 
in how to take notes 
in English? 



English 
Native lang. 
Both 
Another 

English 
Native lang. 
Both 
Another 



Yes 
No 



[If "yes"] How many < 1 hr. 

total hours of class- 1-3 hrs . 

room instruction have 3-5 hrs. 

you had ..." > 5 hrs . 

Have you had any 

classroom instruction 

in how to take notes 

in a language other Yes 

than English? No 

[If "yes"] How many < 1 hr . 

total hours of class- 1-3 hrs. 

room instruction have 3-5 hrs. 

you had. . . " > 5 hrs. 



,79 
.01 
.18 
.00 

.89 
.04 
.08 
.00 



,22 
.78 

.29 
.46 
.09 
.15 



,32 
.68 

.27 
.32 
.12 
.29 



,77 
,01 
.18 
.00 

.87 
.02 
.11 
.00 



.27 

,73 

.19 
.35 
.15 
.32 



,26 
,74 

.30 
.27 
.16 
.27 
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Table 6 presents the academic students' responses to questions about 
note-taking habits and experiences in Ajiierican class lectures. There was very 
substantial agreement that taking notes is more helpful in class lectures than 
in the present test. There was also considerable agreement that taking notes 
in lectures helps one to get better test scores, and that the value of note 
taking in lectures lies in assisting memory, producing notes to study, and 
helping to organize information after the lecture. There was also agreement 
that taking notes helps one to understand a class lecture, but less strong 
agreement that taking notes helps one to listen to the lecture. Only a small 
majority agreed with the statement that there is enough time for note taking 
in lectures. 

Table 7 presents a series of questions about note -taking habits and about 
instruction in note taking received. Students largely reported taking "some" 
or "a lot" of notes in class lectures but "none" or "few" notes in the present 
situation.^ Both in American class lectures (for academic students) and in 
the present situation, English was the predominant language of note taking, 
with some students taking notes in both English and their native language. 
About one -fourth to one -third of the students reported having had any 
classroom instruction in note taking; amounts of reported instruction varied 
widely. (Note that the full text of the seventh question, which paralleled 
the sixth, was "How many total hours of classroom instruction have you had in 
how to take notes in English?" Analogously, the wording of the ninth question 
paralleled that of the eighth.) 



Performance Effects for Student Subgroups 

Although the performance analyses reported earlier showed note taking to 
have no positive effect overall, it was believed that note taking might have a 
positive effect for certain groups of students, as identified by their 
questionnaire responses. One such group consisted of those who are in the 
habit of taking notes. Analyses of variance were conducted to determine the 
influence of note taking (a) for those who answered "a lot" to the question, 
"In your native country, how many notes did you usually take in class 
lectures?", and (b) for those who answered "a lot" to the question, "How many 
notes do you usually take in an American class lecture?" (The second analysis 
involved academic students only.) The dependent variable, as before, was the 
difference between Test A and Test B. Factors were Order and Educational 
Status in the first analysis, and Order in the second analysis. Separate 
analyses were conducted for the Note Taking Allowed and Note Taking Urged 
conditions . 



^Examination of the students' actual note sheets indicated that 75% of 
the students in the Note Taking Allowed condition made at least some 
notations, and 90% of the students in the Note Taking Urged condition did 
so. Analyses of either the quantity or quality of the students' notes would 
be quite complex and were beyond the scope of this study. 
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For the Note Taking Allowed condition, the results indicated no 
significant Order effect in either analysis, showing that note taking had 
little influence on performance, F < 1. For the Note Taking Urged condition, 
however, a significant Order effect in both analyses showed that note taking 
impaired performance, F(l, 117) « 11.73, £<.001, and F(l, 42) « 5.22, 2<.05, 
respectively. Thus, these results parallel those for the entire sample: for 
students who are in the habit of taking many notes, the opportunity to take 
notes was not beneficial, and being urged to take notes was detrimental. 

It was also thought possible that students who have had instruction in 
taking notes might show positive effects of note taking. So analyses of 
variance- -separately for Note Taking Allowed and Urged conditions - -were 
conducted for those who answered "yes" to the question, "Have you had any 
classroom instruction in how to take notes in English?" Analyses were also 
conducted for those who answered "yes" to the question, "Have you had any 
classroom instruction in how to take notes in a language other than English?" 
These analyses were conducted for intensive English students only, since fewer 
than one- fifth of the academic students answered "yes" to these questions. In 
each analysis, the difference between Test A and Test B was the dependent 
variable, and Order was the independent factor. 

In analyses for the Note Taking Allowed condition, the Order effect again 
was nonsignificant, F < 1, showing that note taking did not influence 
performance. In analyses for the Note Taking Urged condition, the Order 
effect was in the direction of a negative influence of note taking in both 
cases, reaching significance in the second case, F(l, 45) - 13.80, p<.001, and 
approaching significance in the first, p = .08. Again, the results generally 
paralleled those observed for the whole sample in showing that being allowed 
to take notes had no effect on performance for students who reported having 
instruction in note taking; urging these students to take notes generally 
Impaired their performance. 

An analysis was also conducted (for the Note Taking Allowed condition) 
for those who reported taking notes in the present si tuat ion- - that is, all 
except those who responded "none" to the question, "How many notes did you 
take on the test today?" (Note that the vast majority of students who 
reported taking notes responded "few" to the above-mentioned question, so that 
a separate analysis for those responding "some" or "a lot" could not be 
performed due to the small number of such students.) Factors were Order and 
Educational Status, and the dependent variable was the difference between 
tests. Once again, the Order effect was nonsignificant, F < 1, indicating 
that the opportunity to take notes did not improve performance, even when the 
analysis was limited to students who reported having taken notes. Other 
effects were also nonsignificant. 

Finally, analyses were conducted for students who agreed, and for 
students who disagreed, with the statements: (a) "Taking notes helped me to 
answer the questions better" and (b) "I felt more at ease when I could take 
notes than when I could not." In each case, the factors were Order and 
Educational Status, and the dependent variable was the difference between 
tests. For the Note Taking Allowed condition, all effects were nonsignificant 
in analyses for those who f^reed, and in analyses for those who disagreed, 
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with each of these two statements. For the Note Taking Urged condition, the 
Order effect was significant, showing a performance impairment due to note 
taking, (a) in analyses for those who agreed with each of these statements, 
£(1,154) - 12.54, £ < .001; F(l,114) « 21.31, 2 < .001, for the two statements 
respectively, and (b) in analyi^es for those who disagreed with each of the 
statements, F(l,161) -=« 13.32, 2 < .001; £(1,102) « 7.17, 2 < -Ol^ 
respectively. Other effects in these analyses were nonsignificant. Thus, 
being allowed to take notes produced no significant benefit, and being urged 
to take notes significantly impaired performance, even for students who 
reported that taking notes helped them answer the questions better or made 
them feel more at ease. These results further support the observation that 
the students' perceptions of the benefits of note taking were not coincident 
with the reality of the situation. 

Questionnaire Data for Student Subgroups 

It is also useful to examine the questionnaire data separately for some 
of the student subgroups just described, to determine if perceptions about the 
value of note taking are linked to certain student characteristics. 

First, it was of interest to examine differences in response according to 
whether students habitually took many notes. For this purpose, students' 
note-taking habits were defined on the basis of their responses to the 
question, "In your native country, how many notes did you usually take in 
class lectures?" (Data from this question were used rather than the question 
about note taking in American class lectures because it reflects long-standing 
note-taking habits, and because the analysis for most questions could include 
the c;:-tire sample and not just the academic students.) Students who responded 
"a lot," who were labeled "frequent note takers," were contrasted with those 
who responded "none," "few," or "some," who were labeled "nonfrequent note 
takers." The data were analyzed separately for the Note Taking Allowed and 
Note Taking Urged conditions. 

For each statement shown in Tables 5 and 6, the number of students 
agreeing with the statement was determined separately for frequent and 
nonfrequent note takers. Then, those instances were identified in which the 
percentage of agreement was significantly different for these two groups at 
the .05 level, according to chi-square analyses. Significant differences were 
found for only two statements. For the Note Taking Allowed condition only, 
frequent note takers disagreed more than did nonfrequent note takers with the 
statement, "The talks were too short for note taking to help me very much," 
X^(l, N « 277) "= 8.84, 2<-01. And for both conditions (in analyses for 
academic students only) , frequent note takers agreed more than did nonfrequent 
note takers with the statement, "Taking notes helps me listen carefully to 
American class lectures," ^^(1, N = 136) =6.20 and H « 127) - 5.64, 

respectively, ^'^•O^. With the exception of these cases, there appears to have 
been little difference between frequent and nonfrequent note takers with 
regard to their opinions about the present note -taking experience or (for 
acade Lc students) their views about note t. iking in American classes. Most 
notably, frequent and nonfrequent note takers differed little in agreement 
with such statements as "Taking notes helped me to answer the questions 
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better" or "I felt more at ease when 1 couM take notes than when 1 could 
not. " 

It was also of Interest to see whether having had formal instruction in 
note taking would be related to the students' questionnaire responses. One 
set of analyses used data from the question, "Have you had any classroom 
instruction in how to take notes in English?" The group that responded "yes" 
was compared to the group that responded "no," with respect to answers to the 
other questionnaire items. Separate analyses were performed for the Note 
Taking Allowed and Urged conditions. Only the intensive English students were 
used in this analysis, since too few of the academic students answered "yes" 
to this question. The two groups did not differ significantly in response to 
any of the statements in Table 5, and the statements in Table 6 were not 
applicable to intensive English students. In an additional set of analyses, 
intensive English students were grouped according to their responses to the 
question, "Have you had any classroom instruction in how to take notes in a 
language other than English," and again no significant differences were found 
between those who responded "yes" and those who responded "no." In general, 
then, there was little evidence of a relation between having had note -taking 
instruction and reactions to the present note -taking experience, at least 
among intensive English students, for whom it was possible co conduct 
analyses . 
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Discussion 



In considering the implications of the results, it is important to 
concentrate primarily on the Note Taking Allowed condition, as this condition 
was the most pertinent to the key Issue under study- -namely , whether 
performance on the TOEFL mini- talks is affected by having the opportunity to 
take notes. A clear finding was that, in the present context, allowing the 
students to take notes had little effect on their performance. 

As discussed in the Introduction, results reported in the literature are 
mixed with regard to note-taking effects in situations comparable to the one 
employed here. Studies have observed a positive effect, no effect, or a 
negative effect, depending on factors related to the testing conditions. One 
factor that appears to be relev^t^ is the pace of the talk, as note taking can 
have an interfering effect when the talk is paced too rapidly to allow 
effective note taking (Peck & Hannafin, 1983). Indeed, in those studies cited 
earlier that obser\'ed a positive effect of note taking, the pace was a 
relatively slow 100 to 120 words per minute. By contrast, the mini- talks in 
the present study, as in the TOEFL test, were presented at 145 words per 
minute, which is a moderate rate of speech (Peters, 1972). The TOEFL mini- 
talks are designed so a student who is proficient in listening comprehension 
should be able to understand whac is said and remember the key information for 
at least a short time thereafter. Nevertheless, while the rate of speech used 
in TOEFL mini- talks is regarded as appropriate for a listening comprehension 
test without note taking, it may be too rapid to permit the benefits of note 
taking (if any) to be realized. 

The finding that urging the students to take notes actually impaired 
their performance may also have to do with the pace of the talks. Because the 
talks were presented at a moderate rate of speech without added pauses, the 
Note Taking Urged condition may have essentially forced many students to be 
writing while they would have preferred to be listening. Students' 
questionnaire responses ^Iso suggested that the pace of the talks may have 
been a factor, as only about one -sixth of the students agreed that they had 
enough time to take as many notes as they wanted. This was not the case only 
for the lower proficiency intensive English students; an additional breakdown 
by intensive English and academic students showed that there was little 
difference between these two groups with respect to the percentage of students 
agreeiiig with the question asking whether there was enough time to take notes 
(e.g., 17% and 18%, respectively, for the Note Taking Allowed condition). By 
contrast, more than half of the academic students reported having enough time 
to take notes in American class lectures. 

Another factor in the lack of note -taking effect may have been the 
relative brevity of the mini i:alks , which lasted between 1-1/4 and 1-2/3 
minutes, compared with 10 to 30 minutes for the studies cited above that have 
found a positive note-taking effect. Granted, less than half the students 
agreed with the statement, "The talks were too short for note taking to help 
me very much." But students' responses may not be wholly accurate in this 
case, particularly given that the question effectively involved a comparison 
with a hypothetical situation- - in this case, a comparison between the present 
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talks and imagined longer talks. From a logical standpoint, it would seem 
that short talks would provide relatively little opportunity for forgetting to 
occur and, thus, not allow note taking to have the effect of counteracting 
forgetting. 

An additional basis for the lack of note- taking effect may have been the 
kinds of questions used. Typically, TOEFL mini-talks are followed by 
questions that tap general understanding or memory for significant 
information. Students are not asked to remember very specific details, such 
as names and dates. Most of the questions deal with information that is 
specifically stated and does not involve a level of detail that is so minor as 
to be difficult to retain without taking notes. That the nature of the 
questions may have been a factor is suggested in the students' widespread 
agreement with the statement, "Taking notes would have helped me more if the 
questions had asked about details such as names and dates" (although, again, 
this involves a comparison with a hypothetical situation) . It would appear 
that the students recognized the memory- aid function of note taking and 
perceived that this function plays a less important role in the absence of 
questions about details that can be readily forgotten. 

Some research supports a seemingly opposing point of view- -that note 
taking is most likely to have a positive effect when the test questions 
involve a higher level of information processing. Einstein et al. (1985) 
found a positive effect of note taking on free recall only for questions about 
high importance- level proposit' ns . And Peper and Mayer (1978) observed a 
positive note-taking effect whua the test called for interpreting the 
information presented but a negative effect when the test involved remembering 
and applying the information. Comparisons among studies are made difficult by 
the differences in testing conditions. (For example, the lectures in the 
above-mentioned studies were several times longer than the mini -talks used 
here.) Nevertheless, a possible hypothesis for further research is that note 
taking can be useful either (a) as an aid to remembering very minor details 
that would otherwise be readily forgotten or (b) as a strategy for remembering 
and interpreting information contained in high importance-level propositions. 

The questionnaire, although intended mainly to ascertain students' 
reactions to the present situation, also obtained useful information about the 
academic students' general note-taking experiences and habits in American 
class lectures. The results suggest that these students have a positive view 
about the value of note taking in the classroom, as the vast majority agreed 
that note taking is helpful, particularly as an aid for remembering lecture 
information and for organizing the information and studying after the lecture 
is over. There was somewhat less (though still majority) agreement that note 
taking facilitates listening and understanding lectures. Furthermore, the 
vast majority of students reported taking some or a lot of notes in lectures, 
and the general preference was to take notes only in English. Particularly 
noteworthy was that a large majority believed note taking to be more helpful 
in American class lectures than in the present situation, a view that was also 
reflected in the relatively low frequency of students who reported taking some 
notes or a lot of notes in this study. 




The students' positive views about classoom note taking are generally 
comparable to those observed by Dunkel and Davey (1989) in a questionnaire 
survey of academic international students. Those authors found an agreement 
rate of at least 83% with statements such as "I find V'v.at I get a better exam 
mark if I take notes," "I find notes taken useful for organizing presented 
material," and "I take notes to have review material for exams." Also, the 
most frequently stated reason why note taking is important was to provide a 
memory aid and/or to remember lecture content. They found that international 
students were quite similar to American students in all of these respects. In 
general, then, the available evidence suggests that international students, 
like American students, feel that note taking is useful in class lectures and 
that its primary value lies in assisting memory. These highly positive views 
about note taking in the classroom stand in contrast with the perceived lesser 
value, and relatively low incidence, of note taking in the present TOEFL mini- 
talks . 



Practical Implications 

The practical question underlying this study is whether it would be 
desirable to allow note taking in the context of the TOEFL mini- talks. Note 
taking apparently did not prove beneficial to performance, as the students' 
mean scores were unaffected by the opportunity to take notes. Furthermore, 
judging from the correlational data, permitting note taking had a minimal 
effect on the relative standing of the students. And note taking did not 
appear to alter the test's reliability. So the study provided no evidence 
that allowing note taking had a beneficial effect on performance or 
substantially changed what was measured by the tesc. These findings suggest 
that there may be little merit in permitting note taking in the TOEFL mini- 
talks . 

On the other hand, one might argue that there could be no harm in 
permitting students to take notes, considering that the opportunity to take 
notes (without constraint to do so) at least did not impair performance. This 
viewpoint appears to receive some support from the students' responses to 
certain items in the questionnaire. A majority of students indicated that 
taking notes helped them to answer the questions better, made them feel more 
at ease, and helped them to remember the information in the talks. Thus, it 
might be reasoned that, in light of the students' apparently favorable 
perception of the note- taking experience, allowing note taking in the TOEFL 
mini- talks might not hurt. 

Although the latter point seems reasonable, the authors believe that, on 
the whole, the weight of the arguments is on the side of not permitting note 
taking in the TOEFL mini- talks. Despite the apparently positive reaction to 
note taking on the part of many students, the fact that allowing note taking 
did not actually improve performance -- even for students who claimed that 
taking notes helped them answer the questions be tter- - shows that the students' 
self -reported perceptions about the benefits of note taking did not accurately 
reflect the reality of the situation. Furthermore, if note taking were 
allowed, with special paper provided, this could send the message that the 
developers of the test regard note taking as useful in this context. In fact, 
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the mini-talks are deliberately structured in a way that minimizes the role of 
factors that might contribute to a beneficial effect of note taking. The 
questions do not cover the kinds of minor details that are forgotten quickly 
if not written down. Also, the talks are relatively short, which effectively 
eliminates both the role of note taking as a means of counteracting 
forgetting, and the need for outlining and organizing points, which only come 
into play with more extended discourse. In short, the current TOEFL mini- 
talks are designed to test comprehension of spoken English rather than memory 
or the ability to organize text, and they are designed in such a- way that 
satisfactory comprehension should lead to successful performance without need 
for note taking. 

If it were thought important to introduce note taking into the TOEFL 
listening comprehension section, in order to simulate a lecture-like 
situation, it would seem most logical to do so in connection with the types of 
talks and test questions that more closely resemble actual classroom lectures 
and tests. In this case, the talks would be considerably longer, which could 
introduce a significant memory factor. Also, the pacing of the talks could 
include pauses to permit more note taking, and at least some of the test 
questions could cover the types of information, such as minor details and 
overall organization of the lecture, that might be most susceptible to note- 
taking effects. 

Even then, one other important practical issue would have to be resolved. 
Allowing note taking on the TOEFL test poses a security problem, in that pairs 
or gro\ips of examinees sitting in proximity could use their notes to 
communicate the answer choices with each other. Test monitors would be unable 
to detect such activities if the notes were written in a language they could 
not read, or if the ■ examinees used a code. Thus, if consideration were given 
to allowing note takii^ig in the TOEFL listening section, it would be essential 
to find a workable solution to this critical security problem. 

Issues for Further Research 

The present study, while providing helpful information regarding note 
taking and TOEFL listening comprehension, calls attention to some additional 
questions to address in further research. One set of questions concerns the 
merits of note taking with a lecture format different from that of the current 
mini- talks. It was suggested above that the lack of a note- taking effect in 
the present study may have been related to such factors as the brevity of the 
present mini- talks, the pacing of the talks, or the types of questions asked. 
These are only hypotheses at this point, however. If a listening 
comprehension subtest were designed that more closely resembled a classroom 
lecture and test, with longer talks, slower pacing, or different types of 
questions, research would be needed to examine the role of note taking in the 
new context. 

Another useful research activity would be to examine the notes taken by 
students in order to identify characteristics of notes that appear to be most 
strongly related to successful performance. Such research would extend the 
work already done on this topic with nonnative English speakers (e.g., Dunkel , 




1986, 1988), in this case with particular application to the TOEFL context. A 
first step could be to examine the notes taken in the present study, since 
analysis of these notes was beyond the scope of the present research plan. 
Even more valuable, however, would be an analysis of notes taken in a talk 
that more closely resembles an academic lecture, considering that such a talk 
may comprise a more appropriate situation in which to take notes. 

It would also be useful to examine differences among major language 
groups in the effects of note taking. The numbers of students per language 
group in the present study were insufficient to permit meaningful comparisons 
(e.g., even the largest group here, the Chinese languages, comprised only one- 
fourth of the sample) . By sampling in a way that yields sufficiently large 
numbers in key language groups, research can more adequately examine 
differences among major language groups in effects of note taking, and in 
note -taking habits and experiences. This research not only could provide 
interesting basic information about group differences, but could also help 
determine whether major language groups would be differentially advantaged by 
the opportunity to take notes in a test of listening comprehension. 
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APPENDIX 



Script of Mini-talk and Questions 



(MA) Questions 1347 refer to the following talk related to fanning, 
(MB) There may be several different ways we can change the ecological balance to 
improve our crop yield. This can be seen in approaches to dealing with the 
plain brown garden snail. Commonly served as an entree or appetizer in many 
restaurants and homes, it is a serious pest in many parts of the United States. 
In our orange and lemon groves, the garden snail feeds on the citrus trees and 
makes the fruit unmarketable. The snail is also fond of home garden 
vegetables, flowers, and ground cover plants. To control these pests, we farmers 
have used chemical baits, but the/ve been both costly and ineffective. Now 
biologists have discovered that some species of flies and beetles found in Europe 
are efficient predators of the garden snail, and they tell us that it may be 
possible to transplant them to American citrus groves. If that proves impossible, 
we may use the smaller, less common mollusk, the decollate [pronounce: dfill ka 
layt] snail that attacks and devours the garden snail. This animal could prove to 
be the best biological control of the garden snail yet discovered. 

13. (MA) What is the main topic of the talk? (12 seconds) 

14. (MA) What problem is the speaker concerned about? (12 seconds) 

15. (MA) What does the speaker say about chemical baits? (12 seconds) 

16. (MA) What might experts do with certain European flies and beetles? (12 

seconds) 

17. (MA) What can be inferred about the future of the decollate [pronounce: dcll 

ka layt] snail? (12 seconds) 

ERiC 3b 



Answer Options in Test Booklet 



13. (A) Types of agricultural pests in the United States. 

(B) Marketability of the brown garden snail. 

(C) Control of the brown garden snail. 

(D) Recent changes in restaurant nenus. 

14. (A) A declining demand for citrus fruit. 

(B) An inability to transplant certain citrus trees. 

(C) The lack of appropriate fertilizers. 

(D) Destruction of crops by snails. 

15. (A) They are extremely effective. 

(B) They have just been developed. 

(C) They cost a great deal. 

(D) They harm fruit and vegetables. 

16. (A) Destroy them. 

(B) Import them to the United States. 

(C) Use them to test chemical pesticides. 

(D) Feed them to garden snails. 

17. (A) It may be eaten in place of the brown garden snail. 

(B) It may prove harmful to citrus plants. 

(C) It may be used to control agricultural pests. 

(D) It may become less costly. 
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